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METHOD FOR CREATING VIDEO PROGRAMS (VARIANTS) 
AND SYSTEM FOR IMPLEMENTING THE METHOD 

Field of the Invention 

5 The present invention relates to television, including interactive television, user 

interfaces, video conferences, telecommunication networks and, more concretely, to a 
method and system for creating video programs, including game show-programs with 
interactive interaction between actors and objects generated by a computer program. 

1 0 B ackground Art 

A method and a system for creating video programs are known, wherein images 
alternate and are show^n m separate windows (window interface, picture in picture). In 
order to do this a participant of a video program is shot with a video camera, an image is 
fonned with the aid of computer with according software and the formed image is 
15 displayed to the participant. The video image and the computer image are displayed to 
viewers sequentially or simultaneously in different windows of one screen, wherein the 
image in one of the windows does not obviously depend on what is in the other. 

Also widely known is a method for creating video programs, in which a user- 
participant of a video program and an image generated by a computer are shot in the 
20 process of interaction of the user with the computer. Wherein, different variants of 
shooting the user are used, which differ by the selection of the angle of approach and the 
montage. When a shot is taken from the direction of the computer, the viewers may 
observe the emotions of the user-participant of the video program, the user's actions, but 
the computer screen and the reaction of computer programs' objects to the user's actions 
25 are not seen. If the shooting is conducted with a show of the computer screen, viewers 
may additionally observe only the back of the viser's head, and do not see his emotions, 
mimics, reactions and gestures. The most widely spread variant is shooting from the side 
when the screen is partially seen, the obtained image on the screen as a rule being of low 
quality, and whereby the user is seen in profile. In other variants of shooting, post- 
30 shooting montage is used, when several pictures shot at different angles of approach are 
combined on a divided screen. In the case of specialized television programs which are 
based on the interaction of a user with objects of a computer program (games, problems, 
tests, teaching), this method is not sufficiently effective. The described approaches do not 
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ensure the display of the objects of the user, the program and the process of their 
interaction on one screen. 

A system is laiown for creating video programs, comprismg a video camera 
coupled to a unit for combining images, a computer with a unit for generating images, also 
5 coupled to the unit for combining images, which has an output for a video signal suitable 
for recording or broadcasting (see the magazine "Seti,*' June 1998, publisher "Otkrytye 
Sistemi," pp. 52-56). The known system makes it possible to create video images in 
accordance with the methods described above. This system does not provide for the 
formation of an image showing the user interacting with objects of the computer program 

10 with which the user interacts in the shooting process. 

A method is known for creating video programs for remotely spaced one from 
another participants (video conference mode), wherein video cameras are used to form a 
video image of users of computers which are interconnected by means of a 
telecommunication network and use, as a rule, one and the same software or work with one 

15 application (see the magazine "LAN," June 1998, v. 4, publisher "Otlcrytye Sistemi," pp. 
81-86). A video image of a participant of the video conference is sent tlii^ough the 
network and is output in a separate window (frame in frame), superimposed on the main 
window in one of the comers. Such a method for remote interaction with the possibility of 
seeing and hearing a partner makes it possible to jointly solve different problems. A 

20 drawback of this method is the visual inconsistency of the user-partner's reaction displayed 
in the separate window with the image on the main screen and with changes of that image. 
The partner of the video conference here acts as an external adviser, and not as a co- 
participant of the actions which are occuning on the main screen. Wherein, if the changes 
take place dynamically and several objects are shown simultaneously, it is not possible to 

25 identify the partner's reaction to a change of separate objects displayed on the main screen- 
Disclosure of the Invention 
The object of the instant invention is to create methods and a system for shooting 
video programs of the type described above, which provide the possibility of shooting a 

30 participant of a video program in the process of the participant's interaction with displayed 
objects of a computer program in such a manner that viewers could watch on a screen the 
participant's reaction to changes in the computer program, the participant's mimics, 
gestures and simultaneously watch the results of operation of the computer program on the 
same screen. 
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The technical result being achieved is enhancement of the quality of the created 
video programs by increasing the reliability and quality of displaying to the viewers 
images fonmed by the computer and the reaction of the actor-participant to a change of 
these miages. Furthermore, the quality of the created images is enhanced by the 
5 possibility which the invention provides for displaying images formed by the computer of 
objects of the computer programs with which a participant of the video program interacts 
in the space between the participant and the viewer, as a result of which the effect of 
viewers' participation in the onrolling performance is created, the impressiveness and 
attractiveness of the video program are enlianced, which promotes an increase of the 
10 viewers' interest in video programs created in this mamier, wherein the video programs 
may be used for a study of the process of interaction of a participant with objects generated 
by the computer. 

The indicated technical result is achieved in that in a method for creating video 
programs, which is based on video shooting and forming computer images, wherein an 

15 image is formed by a computer, which includes an image of objects in the foreground and 
a background image, shooting a participant of the video program is carried out with a video 
camera and a corresponding video image is obtained, an output video signal is formed with 
use of the video image of the participant of the video program and the image formed by the 
computer, in accordance with the invention at least objects of the foreground of the image 

20 formed by the computer are displayed to the participant of the video program, shooting the 
participant of the video program is carried out in the process of displaying at least objects 
of the foreground of the image fonmed by the computer to the participant of the video 
program with an angle of approach of the shooting providing the possibility of reproducing 
in the obtained video image the reaction of the participant of the video program to the 

25 displayed objects of the computer program, the image fonned by the computer is combined 
with the video image of the participant of the video program by superimposing an image of 
at least the objects of the foreground on the video image of the participant, and the 
combined image is used for subsequent display to the user. 

Wherein it is preferable that the shooting of the participant of the video program be 

30 carried out on a chromakey background, and when the image of the objects of the 
foreground, which are fonmed by the computer, is combined with the video image of the 
participant of the video program, the chromakey background is replaced with the 
aforementioned background image or with any other image, wherewith the participant of 
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the video program is provided with the possibility of interacting with the displayed objects 
fornied by the computer, and of changing the image of the aforesaid objects. 

It is also preferable, that at least the video image of a participant of a video 
program, which is shot by a video camera in a studio, and data necessary for foraiing an 

5 image with a computer are transmitted via a telecommunication network to a user device, 
an miage is formed in the user device on the basis of data received from the studio, this 
image including an image of the objects of the foreground and a background image, the 
video miage of the participant of the video program and the image formed by the user 
device are combined by superimposing the miage of the objects of the foreground onto the 

10 video image of the participant, and displaying the combined image to the user. 

Wherein, an input of control commands used to forni the image in the user device 
may be earned out in the user device, the control commands are transmitted through the 
telecommunication network into the studio and are used to form an image by the computer. 
Shooting a user with a video camera, transmitting a video image of the user through 

15 a telecommunication network into a studio, combining the video image of the user received 
in the studio with objects of the foreground by superimposing an image of these objects on 
the video image of the user, and displaying the combined image to a participant of the 
video program, which combined image may be used for display to other users, may 
additionally be carried out. 

20 The indicated technical result is also achieved in that a system for creating video 

programs, combining shooting with a video camera and forming images with a computer, 
primarily television programs, the system comprising a video camera for shooting a 
participant of a video program and a means for forming an image including objects of the 
foreground and a background image, the video camera and the means being disposed in a 

25 studio, in accordance with the invention comprises a means for displaying at least objects 
of the foreground to the participant, the means for displaying being connected to the means 
for fomiing an image, and a means for combining images, a first input of which is 
connected to an output of the video camera, a second input to an output of the means for 
forming an image, wherein the aforesaid means for combining images is made with the 

30 possibility of superimposing an image of objects of the foreground on a video image of the 
participant. It is preferable that the clothes of the participant (participants) be of a neutral 
color or of colors which do not coincide with those of the objects, then the objects will not 
blend with the background (with the participant). 
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Wherewith the means for displaying is preferably made so that the image displayed 
to the participant intersects the line of shooting. In particular, the aforesaid means for 
displaying preferably comprises a screen, coupled to a means for fomiing images and 
disposed outside the limits of the field of view of a video camera, and a semitransparent 

5 muTor, optically conjugated with the aforesaid screen and disposed on the line of shooting 
the participant with the video camera, at an angle to the aforesaid line to provide the 
possibility of forming a reflected image displayed to the participant in a plane substantially 
perpendicular to the line of shooting. 

Furthermore, the system preferably comprises a means for interactive interaction of 

10 the participant with objects of the displayed image, the means being coupled to the means 
for fonning an image, and made, for example, in the fonn of a means for determining the 
position and orientation of the participant. 

In one of the embodiments the system additionally comprises a channel of a 
telecommunication network, a connection unit coupled by two-way communication to the 

15 channel of the telecommunication network, to a means for forming an image and to a 
means for combining images, and at least one user device comprising a user means for 
fomiing an image of the objects in the foreground and a background image, a user 
connection unit coupled by two-way communication to the channel of the 
telecommunication network and to the user means for forming an image, a user means for 

20 combining images of the foreground with a video image of a participant of a video 
program sent over a channel of the telecommunication network and made with the 
possibility of superimposing the image of the objects in the foreground on the video image 
of a participant of the video program and a user means for displaying the combined image, 
wherein a first input of the user means for combining the images is connected to an output 

25 of the user connection unit, a second input is connected to an output of the means for 
fonning images, and an output is connected to an input of the unit for displaying a 
combined image. 

In the indicated embodiment the system preferably also comprises a user control 
unit, an output of which is connected to a corresponding input of the user connection unit, 
30 and/or a user control command processing unit disposed m the studio and connected by 
two-way communication to the connection unit and to the means for fomiing the image. 

The indicated technical result is also provided in that in a method for creating video 
programs in a video conference mode based on video shooting and fomiing images with a 
computer, wherein an image is fomied by computers for each of at least two spatially 
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separated participants of a video program, the image mcluding an image of the objects of 
the foreground and a background image, wherein the aforesaid computers are hnked 
through a telecommunication network, at least the maage of the foreground objects, formed 
by a computer, is displayed to each of the participants of the video program, the possibility 

5 of mteracting with the displayed objects and changing the displayed objects is provided to 
each of the participants of the video program, shooting each of at least two participants of 
the video program is carried out with a video camera, a video image of each participant is 
sent through the telecommunication network and displayed to the other participant, in 
accordance with the invention the shooting of each participant of the video program is 

10 carried out in the process of the participant's interaction with at least the objects of the 
foreground of the image fomied by a computer, the image of the foreground objects which 
are displayed to a particular participant is combined for each of the participants with a 
video image received of another participant of the video program by supeiposmg the 
aforesaid image of the foreground objects on the aforesaid received video image of the 

15 participant of the video program and a combined video image is displayed to each of the 
participants of the video program, wherein the background, on which a participant of the 
video program is shot with a video camera, is cleaned from the received video image and 
replaced with a background image formed by a computer or by any other image. 

Furthemaore, the technical result indicated above is achieved in that in a method for 

20 creating video programs for registering the reaction of a user to a presented image for 
studying and optimizing interfaces of computer programs and editing video films, wherein 
an image is formed, the fomied image is displayed to the user, the user is shot with a video 
camera and a corresponding video image is obtained, an output video signal is formed with 
use of the video image of the user and the formed image for subsequent analysis, in 

25 accordance with the invention, the shooting of the user is carried out in the process of 
displaying the formed image to the user with an angle of approach of the shooting which 
provides the possibility of reproducing m the obtained video image a reaction of the user to 
the image displayed to the user, transparent zones are created in the image being formed, 
the fomied image is combined with the video image of the user by superimposing the 

30 formed image with the transparent zones on the video image of the user. 

Wherein, the image formed by the computer comprises an image of the objects and 
a background image, the background image being fonned transparent when combined with 
the video image of the user. 
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Furthermore, the user is provided with the possibiUty of interactuig with the objects 
of the displayed image fonned by the computer. 

It is useful to additionally register data of the psychophysiological condition of a 
user in the process of displaying the fomied image to the user, wherein the indications of 
5 the registered data are combined with the images of the video program. 

Brief Description of the Drav/ings 
The mvention is explained with examples of embodiment illustrated with drawings. 
Fig. 1 shows a block diagram of a system for creating video programs, m 
10 accordance with the invention. 

Fig. 2 shows embodiments of realizing the means for displaying an image fomied 
by a computer to a participant of a video program. 

Fig. 3 shows a block diagram of a system for creating video programs, which 
provides the possibility for interactive interaction of a user with a participant of a video 
15 program and/or with objects generated by a computer. 

Fig. 4 shows a block diagram of a system for creating video programs in a video 
conference mode. 

Embodiments of the Invention 

20 A system for creating video programs, primarily TV programs, shown in Fig. 1, 

comprises a video camera 1, serving to shoot a participant 2 of a video program, a means 3 
for fomiing an image to be displayed to the participant 2 with the aid of a means 4 for 
displaying, a unit 5 for combining images, one input of which is connected to an output of 
the video camera 1, another input - to an output of the means 3 for forming an image, and 

25 an output of the unit 5 for combining images is an output of a finally formed video signal 
which may be used for transmission to user terminals (not shown in Fig. I). An input of 
the means 3 for forming an image is connected to an output of a means 6 for interaction of 
a participant 2 of a video program with objects foimed by the means 3 (computer). All 
possible manipulators and sensors - mouse, joystick, keyboard, transparent sensor screen, 

30 virtual gloves, game consoles, sensors of body part movements, microphone for receiving 
vocal commands, may be used as the means 6 for interaction. 

Embodiments of reahzation of the means 4 for displaying an image formed by the 
means 3 are shown m Fig. 2 (a)-(d). Fig. 2 (a) shows a screen 7, for example, a monitor 
screen on which images formed by the means 3 are displayed. The screen 7 is set in such 
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a manner that it does not obstruct the shooting of a participant, i.e. is outside the field of 
view of the video camera. A glass plate 8 (or a semitransparent mm-or), foiming a false 
(or real) image T is mounted on the shooting line 0-0', passing through the video camera 1 
and the participant 2, at an angle to the line 0-0', preferably at an angle of about 90 
5 degrees to the line 0-0', m accordance with how the display of a standard display means is 
oriented. 

An embodiment of the means 4 for displaying is shown in Fig. 2 (b) for the cases of 
shooting the participant at full height (view from above). In this embodiment the means 4 
for displaying comprises a screen 7, mounted outside the field of view of the video camera 
10 1, a semitransparent miiTor 8, mounted on the shooting line O-O', and a projector 9, 
coupled to a means 3 for fomiing an image and optically conjugated with the screen 7. 

Embodiments are shown in Figs. 2 (c) and (d) for realization of the means 4 for 
displaying for outside-studio use. hi the variant according to Fig. 2 (c), the means 4 for 
displaying is a standard monitor 10, on which a small-size video camera 1 1 is mounted, put 
15 forward of a monitor 10 screen and providing a video conference mode. The binocularity 
of a user's sight ensures that the video camera 11 will not close any of the objects of the 
image simultaneously for both eyes of the user, hi the variant according to Fig. 2 (d) a 
notebook monitor 12, positioned in the same plane as its keyboard 13, serves as the display 
means 4. The image from the monitor 12 is displayed to the user through the plane 8, thus 
20 creating a false image 12'. 

The system for creating video programs, w^hich is shown in Fig. 3, provides for the 
transmission of created television programs through telecommunication networks and 
provides the possibihty for interactive interaction between user-viewers and the objects of 
the video program and/or the participant, hi the system according to Fig. 3, a second 
25 output of the unit 5 for combining the images is coupled to the telecommunication network 
14 through a corresponding connection unit 15, for example, modem. The means 3 for 
forming an image is coupled by two-way communication to the connection unit 15. The 
telecommunication network 14 is connected to a user device 16 which comprises a 
corresponding connection unit, the functions of which m the variant under consideration 
30 are performed by a processor device 1 7, outputs of which being connected to an input of a 
user means 18 for forming an image and to one of the inputs of a user unit 19 for 
combining images. An output of the foming means 18 is connected to another input of 
the unit 19 for combining images, an output of which is connected to an input of a unit 20 
for display of a combined image. The user device 16 also comprises a control unit 21 
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connected to the input of the processor device 17 to input the user^s control commands, 
which may be sent through the telecommunication network 14 and the connection unit 15 
into the studio. 

In the embodiment of the claimed system, which provides for transmission of a 

5 signal of a combined image from the studio through the air, through a cable or through 
other channels, the system comprises a corresponding receiving device with an antenna 22. 
In this case a combined image and data on the parameters of an image foimied by a 
computer in the studio come through the air to the user. Using this data at the user device 
16, the user may interact, using the control unit 21, with objects meant to be controhed by 

10 the user. Wherein, a return signal from the user, containing control commands, may be 
sent to the studio through the telecommunication network 14. 

In the embodiment of the system, which is presented in Fig, 3, the system also 
comprises a unit 23 for processing the user's control commands, which unit is coupled by 
tw^o-way communication to the connection unit 15 and to the means 3 for foming images. 

15 Users' control commands may be used to obtain individual infonnation from the users: to 
control objects generated with the aid of computer software, to detemiine users' ratings, to 
distribute opinions of groups of users, their preferences, individual evaluation of users' 
actions, organization of a competition between an actor and one or several distant users. 
In accordance with information concerning the users, individualized special inforaiation 

20 may be sent to the user devices: tasks, advertisements, evaluation, proposals, results of 
processing generalized mfomiation from all of the users. Such individual information 
may be displayed for each user m separate windows of an image common to all. 

The system for creating video programs in a video conference mode, shown m Fig. 
4, provides the possibility for several partners to see each other during interaction with 

25 objects of computer programs, which are displayed to them on a background of a video 
image of the partner. The system comprises video cameras 1,1' for shooting participants 
2, 2', respectively, means 3, 3' for forming an image with a computer, means 4, 4' for 
displaying video images received from the network 14 and formed images to participants 
of a video conference, units 5, 5' for combining video images of participants of the video 

30 conference and the formed images, and units 15, 15' for connecting to the network 14, 
which are coupled to the units 5, 5' for combining and to the means 3, 3' for forming an 
image. 

The system for creating video programs, made in accordance with the invention, 
operates in the following manner. 
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A participant 2, who is shot by at least one television camera 1, is placed m a 
television studio as shown m Fig. L As a rule, the participant is shot full face, sitting at a 
table, as hosts of television programs are usually shot, or in full height, providing a 
participant with the possibility of moving in a hmited space. Wherein, simultaneously 
5 with the shooting, images are displayed to a participant of the video program which are 
formed by the means 3, for example, a computer game. The video image of the 
participant, obtained by the video camera, and an image from the means 3 for forming are 
combined in the unit 5 for combining images. The images formed by the means 3 are 
actually the result of operations canied out m a computer by a corresponding program and, 
10 as a rule, consists of tw^o parts (tw^o layers): a background image and objects. The 
background image and the objects, in turn, may themselves also consist of several layers. 
With the aid of a corresponding program unit, any layer may be separately output. The 
image of the objects is separated from the background image and in the unit 5 for 
combining is superimposed on the video image of the participant, which is formed by the 
15 video camera. Both the image of the objects and the background image made be made 
semitransparent, ahemating pixels of the image and transparent pixels. When such an 
image is superimposed on the video image of a user, the latter will be seen by viewers 
through the image formed by the computer. If the images are combined by television 
analogue mixers, then those parts of the image being formed by the computer, which 
20 should be transparent, are filled with a chromakey color so as to use the rear-projection 
method during superimposition. In that case the chromakey color is removed, and the 
place which it occupied m the image becomes transparent. Combining the images is 
carried out in such a manner that a viewer sees a participant watching the objects fonned 
by the means 3. When the images are combined any of the layers of the images may be 
25 made semitransparent, i.e. the degree of its transparency may be changed from zero to one 
during the solution of certain problems. Such an adjustment of the degree of transparency 
of one of the layers may be accomplished in the studio by a director or in a user device by 
a user. Wherewith, a viewer, watching the combined image, gets the impression of 
participation. It seems to the viewer that the objects fonned by the computer and 
30 displayed on the screen are between the viewer and the participant of the video program. 
So that the means 4 for displaying would not interfere with the shooting of the participant 
and would not fall within the field of view of the video camera, the means 4 is made as 
shown in Figs. 2 (a)-(d). The images formed by the means 3, which is outside the field of 
view^ of the video camera, are displayed to the parficipant, who is a user of the means 4 for 
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displaying, in such a manner that the images would not block the television camera from 
the participant being shot. 

As shown in Fig. 2 (a), at least an image of the objects formed by the means 3 is 
displayed on the screen 7. The glass plate (or semitransparent mirror) 8 mounted above 
the screen 7 (at the side or above the screen) provides for fomiing an image 1\ which is a 
reflection of the screen 7, transverse to the axis 0-0" passing along the line of shooting a 
participant (or participants) of the video program. In this case, a participant 2 watches the 
image-reflection T of the image from the screen 7. Wherein, the Applicant's gaze will be 
simultaneously directed to the screen T and towards the video camera. If the image 
formed by the computer on the screen 7 is temporarily turned off, the participant may 
concentrate his gaze on the lens of the television camera as in traditional shooting. Taking 
into account the dimensions of the screen 7, the distance from the plane on which the 
image T is formed to the participant 2, the distance from the participant 2 to the video 
camera 1 and its parameters, and the laws of perspectivity during the creation of images, it 
is possible to ensure exact coincidence of the participant's reaction to the image T m the 
combined image for television viewers (for example, directing his gaze to an object or 
moving a hand over the sensor screen after the object being displayed). An image 
containing a caption is corrected m such a manner that the captions on the combined 
image, displayed to the viewers and displayed to the participant m the plane of forming the 
image 7', would look natural. 

When a participant is shot m full height (Fig. 2 [b] - view from above) an image on 
a large translucent screen 7 is formed with the aid of the projector 9, which is connected to 
the means 3 for fonning an image. A participant 2 watches the image 1\ an image from 
the screen 7 reflected from the semitransparent mirror 8. 

If the creation of video images is meant for one or a few users, the video 
conference mode may be realized, wherein users do not only see each other but may 
control the objects which are displayed to them and which they observe between 
themselves and a partner. In accordance with Fig. 2 (c), the video conference mode is 
realized with the use of a computer and a video camera, wherein software provides for the 
display of a partner's image on a background of objects generated by the means for 
forming images (by a computer), not m a separate windov/. Partners, participating m the 
video conference, see one and the same objects m front of them, each from its own side, 
and they may jointly discuss these objects or introduce changes therein. Such a variant of 
realizing video conferences is especially effective in the case of playing computer games 
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with a partner or as a computer variant of realization of "table" games. It will seem to the 
partners that they are playing at one table, they will see and hear each other, see each 
other's reaction to coiTcsponding moves, even though at that moment the partners will be 
separated by thousands of kilometers. In outside-the-studio conditions, it is also easy to 
realize the variant described above with reflection of an image from the semitransparent 
miiTor 8 with use of a "notebook" type computer (Fig. 2 [d]). 

The embodiment of the system providing for interactive interaction between a 
viewer with objects generated by a computer and/or with a participant (Fig. 3) operates m 
the following manner. A signal, containing a video image of a participant and data 
necessai-y to form an image with a user's computer, is sent from the studio through the 
telecommunication network 14 to a user device 16. A user means 18 for forming fornis 
an image of the objects. Then, using the user unit 19 for combining, the video image of a 
participant of a video program, which image is sent over the network, and the image of 
objects, which is formed by the means 18, are combined, putting the image of the objects 
over the video image of the participant. Then the combined image is displayed to a 
viewer-user with the aid of the means 20 for displaying. If the viewer-user uses a control 
unit 21, then the means 18 for fonning forms an image of the objects, taking into account 
the data received from the studio and the control commands from the unit 21 . Wherewith, 
the control commands are sent to the studio and are processed in the studio by the unit 23. 
Different variants for providing a viewer with interactive interaction with objects fomied 
by a computer are possible. For example, back coupling to a studio through a 
telecommunication network may be used. Without use of such back coupling, the viewer 
will receive the combined image and data on the image fonned by the computer through, 
for example, the air or by cable. The means necessary in the majority of cases for 
synclironizmg, digitizing, archiving, compressing images and for reverse operations are not 

shown m the drawings. 

The system for creating video programs in a video conference mode (Fig. 4) 
provides users remotely spaced one from another with the possibility to obsei-ve objects 
foi-med by a computer, as if between each other. In order to do this the video image of one 
user 2, shot by a camera 1, is sent through a telecommunication network 14 to be combined 
with objects formed by a means 3' of another user 2', after which the objects formed by the 
means 3' are superimposed on the video image of the user 2 in the unit 5' for combining the 
images. The same operations are simultaneously carried out for the other user 2'. 
Wherein, the image formed by the computer is made transpai-ent over some areas and is 



superimposed on the video image of the other user. Each user will thus see the objects 
formed by the computer between himself and the partner of the video conference. 

hi a preferable embodiment of the invention, a video camera is used in the studio, 
the video camera shooting through a semitransparent mirror (glass plate) 8 w^hich is 
5 mounted at an angle of 45 degrees to the shooting line O-O' and reflects to the participant 
the screen of a monitor on which a computer game, which the participant is playing during 
the shooting, is displayed. Wherewith, the shooting of the participant may be carried out 
on a chromakey background, and the display of the objects of the computer game and the 
participant - on a background formed by that same program which fonns the objects of the 

10 computer program, or on any other background which may be changed at any moment of 
the shooting. Combining the objects formed by the computer with the video image of the 
user is carried out in a real-time mode. Wherein, the objects formed by the computer are 
superimposed on the video image of the participant. And if the participant is shot on a 
chromakey background, then, in turn, the video image of the participant (with an already 

15 transparent background) is superimposed on the background image. 

Traditional computer and video games may be used as the software used m the 
shooting, hi order to do this, small changes, providing for the separate output of an image 
of the objects and a background image, should be introduced into those games. 

It should be noted that the concept "video programs," which is used in the instant 

20 specification, should be understood to be wider than programs for standard television 
broadcastmg. Video programs, created by the method according to the instant invention 
may be inserted into computer networks and transmitted through computer networks, 
including with the presentation of the possibility for interactive interaction with a "viewer." 
The claimed method may be used to organize video conferences, create educational video 

25 programs and video programs for studying the reaction of users. 

Use of the claimed method is especially useful for malting a study and optimization 
of user interfaces of computer programs so that the programs would be intuitively 
understood and convenient for users. It is possible to track and time all the actions of a 
user while working with a computer with a program, a certain interface, and a complete 

30 record of his actions. The user himself cannot control and consciously describe all his 
actions. Quite often these actions are carried out subconsciously. Many reactions are not 
controlled by the user himself, for example, small movements of his eyes and reflex 
reactions. Special sensors may be used to register the area tow^ard which the user's gaze is 
directed, and this area will be displayed on a con-esponding place in the image of the 
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objects. A person's eye may see clearly and sharply only a very small area which is in the 
center of the angle of view, the so-called Foil zone. The image as a whole seems to a 
person to be sharp due to unconscious micromovements of the eye. Wherein, a person's 
sight especially keenly reacts to changes in the image being observed, constantly sending 
5 information thereon into the brain and receiving in response control commands at the 
subconscious level. In the video programs created in accordance with the present 
invention, the majority of a user's reactions, conscious and unconscious, will be registered. 
Further improvement of computer programs, taking the results of studies of video materials 
into account, will aid to increase the volume of the sale of these programs, to increase their 

10 competitiveness in the market, as a resuh of selection of optimum functions and user 
interface. Optimization of the interface and the structure of a program in tum will save the 
time and simplify the work done by a final user, increase the productivity of his labor. 
Further advantage during an examination of video protocols (video programs) of the user's 
work with a computer may be provided by use of sensors of a user's eye movements, 

15 emotional reaction etc., and a corresponding input of their indications to the video 
program. 

Industrial Applicability 
The invention corresponding to the claimed method may be used for medical and 

20 professional testing and training of users. In the case of specialists in psychophysiological 
testing, the possibility is provided for registering not only the actions of those being tested, 
but also to study the reaction of users, including subconscious reaction recorded in the 
form of a video program. The reactions of users on a background of stimulator images 
displayed to them and data on the dynamic monitoring of the state of users being tested are 

25 recorded in the video program. Wherewith, monitoring of the pyschophysiological 
condition of the user being tested may be carried out with different types of sensors: a 
sensor of the emotional condition ("he detector," polygraph), blood pressure, pulse, and 
they may be displayed on the same screen. Such a method may be more widely used, not 
only when a user works with a computer, but also during a study of a viewer's reaction 

30 while the viewer is watching a movie or an animated cartoon. For this, the image of a 
video film is reproduced instead of an image generated by a computer. The combined 
results of a study of the reactions of a representative selection of viewers to a certain video 
film in accordance with the invention, actually provide an objective evaluation of the video 
film itself on the basis of objective psychophysiological data. A study of the reactions of 
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viewers in control groups prior to releasing the film for wide demonstration makes it 
possible to obtam data for analysis of the reaction of the mass of viewers and to more 
exactly chose the best variants for development of the plot and for editing to obtain 
predictable results. Achievement of the necessary indications, in accordance with the 
results of the studies, will help to improve the drawing power of films and to obtain the 
necessary reaction of the mass viewer. Where necessary, infoniiation transmitting the 
reaction of a viewer to a separate concrete frame may be obtained, this making it possible 
to track the movement of a viewer's gaze over the plane of a displayed image. 
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Set of Claims 



1 A method for creating video programs, which is based on video shooting and 
forming computer images, wherein 
5 an image is fornied by a computer, wliich includes an miage of objects in the 

foreground and a background image, 

a participant of a video program is shot with a video camera and a coirespondrng 
video image is obtained, 

an output Video signal is foniied with use of the video image of the participant of 
10 the video program and the image formed by the computer, 
characterized in that 

at least objects of the foreground of the image formed by the computer are 
displayed to the participant of the video program, 

shooting the participant of the video program is carried out in the process of 
15 displaying at least objects of the foreground of the image fomied by the computer to the 
participant of the video program with an angle of approach of the shooting providing the 
possibility of reproducing in the obtained video image the reaction of the participant of the 
video program to the displayed objects of the computer program, 

the image fomied by the computer is combined with the video image of the 
20 participant of the video program by superimposing an image of at least the objects of the 
foreground on the video image of the participant, and the combined image is used for 
subsequent display to a user. 

2. A method according to claim 1, characterized in that 

the shooting of the participant of the video program is carried out on a chromakey 
25 background, and when the image of the objects of the foreground, which are fomied by the 
computer, is combined with the video image of the participant of the video program, the 
chromakey background is replaced with said background image or with any other image. 

3. A method according to claim 1, characterized in that the participant of the video 
program is provided with the possibility of interacting with the displayed objects formed 

30 by the computer and of changing the image of said objects. 

4. A method according to any one of claims 1-3, characterized in that at least the 
video image of a participant of a video program, which is shot by a video camera m a 
studio, and data necessary for forming an image with a computer are transmitted via a 
telecommunication network to a user device, 
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an image is formed in the user device on tlie basis of data received from the studio, 
this image inckiding an image of the objects of the foreground and a background miage, 

the video image of the participant of the video program and the image formed by 
the user device are combined by superimposing tire image of the objects of the foreground 
5 onto the video image of the participant, 

the combined image is displayed to the user. 

5 . A method according to claim 1 , characterized in that 

control commands are input to the user device and the obtained control commands 
are used to form the image in the user device, the control commands are transmitted 
10 through the telecommunication netAvork into the studio and the control commands received 
in the studio are used while fonuing an image with the computer. 

6. A method according to claim 5, characterized in that 
shooting a user is effected with a video camera, 

a video image of the user is transmitted through a telecommunication network to 
15 the studio, 

the video image of the user received in the studio is combined w4th objects of the 
foreground of the image fornied by the computer in the studio by superimposing an image 
of said objects of the foreground on the video image of the user, 

the combined image is displayed to a participant of the video program. 
20 7. A method according to claim 6, chajracterized in that 

the combined video image of the user and the objects of the foreground of the 
image formed by the computer is used for display to other users. 

8. A system for creating video programs, combining shooting with a video camera 
and foiming images with a computer, primarily television programs, the system 
25 comprising 

a video camera for shooting a participant of a video program and 

a means for forming an image including objects of the foreground and a 
background image, 

the video camera and the means being disposed in a studio, 
30 characterized m that it comprises 

a means for displaying at least objects of the foreground to the participant, the 
means for displaying being connected to the means for fondling an image, and 

a means for combining images, a first input of which is connected to an output of 
the video camera, a second input to an output of the means for forming an image, wherein 
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said means for combining images is made with the possibility of superimposmg an image 
of objects of the foreground on a video image of the participant. 

9. A system according to claim 8, characterized in that said means for displaying is 
made so that the image displayed to the participant intersects the line of shooting the 

5 participant with the video camera. 

10. A system according to claim 9, characterized in that said means for displaying 
comprises 

a screen, coupled to a means for forming images and disposed outside the limits of 
the field of view of the video camera, 
10 a semi transparent mirror, optically conjugated with said screen and disposed on the 

line of shooting the participant with the video camera, at an angle to said line to provide 
the possibility of foming a reflected image to be displayed to the participant in a plane 
substantially perpendicular to the line of shooting. 

1 1. A system according to any one of claims 8-10, characterized in that it comprises 
15 a means for interactive interaction of the participant with objects of the displayed image, 

the means being coupled to the means for forming an image. 

12. A system according to claim 11, characterized in that said means for interactive 
interaction is made in the form of a means for detenmining the position and orientation of 
the participant. 

20 13. A system according to any one of claims 8-12, characterized in that it comprises 

a channel of a telecommunication network, 

a connection unit coupled by tw^o-way communication to the channel of the 
telecommunication network, to the means for forming images and to the means for 
combining images, 
25 at least one user device comprising 

a user means for fomiing an image of the objects in the foreground and a 
background image, 

a user connection unit coupled by two-way communication to the channel of the 
telecommunication network and to the user means for forming an image, 
30 a user means for combining images of the foreground with a video image of a 

participant of a video program, sent over the channel of the telecommunication network, 
and made with the possibility of superimposing the image of the objects in the foreground 
on the video image of a participant of the video program, and a user means for displaying 
the combined image. 
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wherein a first input of the user means for combining the images is connected to an 
output of the user connection unit, a second input is connected to an output of the means 
for forming images, and an output is connected to an input of the unit for displaying the 
combined image. 

14. A system according to claim 13, characterized m that it additionally comprises 
a user control unit, an output of which is connected to a corresponding input of the user 
connection miit, and a user control command processing unit disposed in the studio and 
connected by two-way communication to the connection unit and to the means for forming 
an image. 

15. A method for creating video programs m a video conference mode, based on 
video shooting and forming images with a computer, wherein 

an image is formed by computers for each of at least two spatially separated 
participants of a video program, the image including an image of objects of the foreground 
and a background image, wherein said computers are linked through a telecommunication 
network, 

at least the image of the foreground objects, formed by a computer, is displayed to 
each of the participants of the video program, 

the possibility of interacting with the displayed objects and changing the displayed 
objects is provided to each of the participants of the video program, 

shooting each of at least two participants of the video program is carried out with a 
video camera, 

a video image of each participant is sent through the telecommunication network 
and displayed to the other participant, 
characterized in that 

the shooting of each participant of the video program is carried out in the process of 
the participant's interaction with at least the objects of the foreground of the image fomed 
by a computer, 

the image of the foreground objects which are displayed to a particular participant 
is combined for each of the participants with a received video image of another participant 
of the video program by superimposing said image of the foreground objects on said 
received video image of a participant of the video program and 

a combined video image is displayed to each of the participants of the video 
program. 
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16. A method according to claim 15, characterized in that the background, on 
which a participant of the video program is shot with a video camera, is cleaned from the 
received video image and replaced with a background image formed by a computer or with 
any other image, 

5 17. A method for creating video programs for registering the reactions of a user to 

an image displayed to a user for studying and optimizing interfaces of computer programs 
and editing video films, wherein 
an image is fomied, 

the foimed image is displayed to the user, 
10 the user is shot with a video camera and a correspondmg video image is obtained, 

an output video signal is fonned with use of the video image of the user and the 
formed image for subsequent analysis, 
characterized in that 

the shooting of the user is carried out iii the process of displaying the fonned image 
15 to the user with an angle of approach of the shooting which provides the possibility of 
reproducing in the obtained video image a reaction of the user to the image displayed to the 
user, 

transparent zones are created in the image being formed, 

the fomaed image is combined with the video image of the user by superimposing 
20 the fomied image with the transparent zones on the video image of the user. 

18. A method according to claim 17, characterized in that the image is formed by a 
computer, wherein the image comprises an image of the objects and a background image, 
the background image being formed transparent when combined with the video image of 
the user. 

25 19. A method according to claim 18, characterized in that the user is provided with 

the possibility of mteracting with the objects of the displayed image fonned by the 
computer. 

20. A method according to any one of claims 17-19, characterized in that data of 
the psychophysiological condition of a user in the process of interaction with the displayed 

30 objects of the image formed by the computer are additionally registei'ed. 

21. A method according to claim 20, characterized in that the indications of the 
registered data are combined with the images of the video program. 
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Abstract 

The invention relates to television, mteractive television, user interfaces, video 
conferences and may be used in the creation of video programs with interactive interaction 

5 between participants of video programs and objects generated by a computer program. 
The technical result is enhancement of the reliability and quality of the display to viewers 
of images, formed by a computer, and the reaction of participants of a video program to a 
change of the images. A participant 2 of a video program is shot by a video camera 1, 
wherein an image, fonned by a means 3 (computer), is simultaneously displayed to the 

10 participant 2 on the line of shooting with the video camera. At least an image of objects of 
a forward plan, which is fomied by the means 3, is combined in the unit for combining 
images, these objects being put on the video image of the participant, obtained by the video 
camera 2, in such a manner, that a viewer sees the participant 2, who is watching the 
objects of the forward plan and interacting with them with the aid of a means 6. 
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I hereby claim the benefit under 35 U.S.C. § 120/365 of any United States application(s) 
listed below and PCT international applications listed above or below: 

PRIOR U.S. OR PCT APPLICATIONS 

Application No. ^series code/serial no,) Month/Day/Year Filed Statusfpending, abandoned, patente d) 



I hereby appoint: 



TIMOTKV G ACKERMANN, Reg No,._44,491„ 
THOMAS E ANDERSON, Reg No^JM^-^- 
BENJAMIN J BAI, Reg No43;^48i — 
MICHAEL J BLANKSTErisrrR^iNo.JZQSiZ_-^ 



MARY JO BOLDiNGH, Reg No^J4JJJ_ 
MARGARET A. BOULWARE, Reg N o. 28.708 

ARTHUR J BRADY, Reg No^42a^6 

MATTHEW O BRADY, Reg" No_a£UiS-4_ 
DANIELJ BURNHAM, Reg Nq^9A13„ 
THOMAS L CANTRELL, Reg NoJO^M?— 
RONALD B COOLLEY, Reg. No^JJJAl 
THOMAS L CRJSMAN, Reg No_2iU846 
STUART D DWORX. Reg NoJLip3 
WILLLAM F ESSER. Reg N a 38.Q53 _ 
ROGER J FRENCH, Reg No.a2a&6 — 
JANET M. GARETTO, Reg. N o 42^ 6S — 

JOHN C GATZ, Reg Nq_1LZZ4 

RUSSELL J GENXT, Reg No^ZS2±^ 
J KEVIN GRAY, Reg No JTJUU- — 



STEVEN R GREENFIELD, Reg No 38,166 
J PAT HEPTIG. Reg No^jg^3.„™- 
SHARON A ISRAEL, t^egNQ^jJ^f^? 
JOHN R KIRK JR., Reg No24477, 
PAUL R KITCH, Reg NoJ^S^Se 
TIMOTHY M KOWALSRi, Reg No^_44J92 
JAMES F LEA III, Reg N o. 41. 143 
ROBERT W MASON, Reg. No _42^8_ 
ROGER L. MAXWELL, Reg Na^,855, 
ROBERTA McFALL. Reg No 3968 
LISA H MEYERHOFF. Reg. No 36,869 
STANLEY R MOORE, Reg. Nc'I^jS 
RICHARD J MOURA. Reg. No;^^^^ 
MARK V MULLER. Reg Na_ 3,7.509 
P WESTON MUSSELM AN JR Reg No 31.644^ 
SPENCER C, PATTERSON, Reg No. 43JJ9 
RUSSELL N. RiPPAMONTI, Reg No.J9^L 
STEPHEN G RUDISILL., Reg No^ 
HOLLY L RUDNICK, Reg No. 43,065 



J L. JENNIE SALA2AR, Reg N o._45.065 
KEITH W SAUNDERS. Reg No^JtL462 
JERRY R SELINGER, Reg N o ^26.58 2 
KEVIN J SIMONS. Reg NoJUUo' 
BOBBY D SLATON, Reg No_43...1.30 
GARY B. SOLOMON. Reg No 44,34 7 
WAYNE O STACY, Reg N o^45J25 " 
SCOTT B, STAHL. Reg N o 33.795 
ROBERT C STRAWBRJCH, Reg No 
STEVE Z SZCZEPANSKL Reg No 2 7.957 
ANDRE M SZUWALSKI. Reg No~S2Hrr 
ALAN R. THIELE. Reg ^oJO,^^ 
WILLIAM J TUCKER. Reg No.,ALa5^ 
T AM SEN V A LOIR. Reg No_4Mn__^ 
RAYMOND VAN DYKE, Reg No _ 34.746 

BRIAN D WALKER, Reg. No^.7_5j ^ 

GERALD T WELCH, Reg NoJ0J12_ * 
HAROLD N WELLS, Reg No^J^MiB^^ 
WILLIAM D WIESE. Reg No 45.217 
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all of the firm of JENKENS & GILCHRIST, P.C., 3200 Fountain Place, 1445 Ross Avenue, 
Dallas, Texas 75202-2799, as my attorneys and/or agents, with full power of substitution and 
revocation, to prosecute this application, provisionals thereof, continuations, continuations-in-part, 
divisionals, appeals, reissues, substitutions, and extensions thereof and to transact all business in 
the United States Patent and Trademark Office connected therewith, to appoint any individuals 
under an associate power of attorney and to file and prosecute any international patent application 
filed thereon before any international authorities, and I hereby authorize them to act and rely on 
instructions from and communicate directly with the person/assignee/attorney/firm/organization 
who/which first sent this case to them and by whom/which I hereby declare that I have consented 
after full disclosure to be represented unless/until I instruct them in writing to the contrary. 
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Please address all correspondence and direct all telephone calls to: 
Stanley R. Moore 
^ J enkens & Gilchrist, P.C. 
_32Qa£oiiatiLinJ^^ 

Dallas, Texas 75202-2799_ 
14/855-47 1 3 2Ul855^3001^x) 

I hereby declare that all statements made herein of my own knowledge are trae and that all 
statements made on information and belief are believed to be true; and further that these statements 
were made with the knowledge that willful false statements and the like so made are punishable 
by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code, and 
that such willful false statements may jeopardize the validity of the application or any patent issued 
thereon. 



NAMED INVENTOR(S) 



1 


LATYPOV Klurakhmed 
Nurislamovicn 

Full Name 


Inventor's Signature 


16.02.2001. 
Date 


Moscow, Russia '''^^Ij^s^ Russia 
Residence (city, state, country) Citizenship 


Russia, Moscow, 5 Voikovsky proezd, d. 10, kv, 31 
Post Office Address (include zip code) 12 5171 


2 


LATYPOV Nurulla 
Nurislamovich 

Full Name 


Inventor's Signature 


16.02.2001. 
Date 


Moscow, Russia ^^^)j[)>^ Russia 
Residence (city, state, country) Citizenship 


Russia, Moscow, 5 Voikovsky proezd, d. 10, kv. 31 
Post Office Address (include zip code) 125171 



T 



